Motion compensated inter-frame coding has been widely adopted in various coding standards, such as MPEG-1/2/4 and H.261/H.263/H.264/AVC. Motion estimation and compensation as well as subsequent processing are performed on a block basis. During compression process, coding noises may arise due to lossy operations applied such as quantization. The coding artifacts may become noticeable in the reconstructed video data, especially at or near block boundaries. In order to alleviate the visibility of coding artifacts, a technique called deblocking has been used in newer coding systems such as H.264/AVC and the High Efficiency Video Coding (HEVC) system under development. The deblocking process applies filtering across block boundaries adaptively to smooth the large transitions at and near block boundaries due to coding noises while retaining image sharpness. Furthermore, due to the nature of inter-frame coding, the deblocking process is configured for in-loop operation. In the recent HEVC development, adaptive loop filtering (ALF) is being adopted to process deblocked reconstruction frames. Adaptive loop filtering is used as in-loop processing in addition to deblocking and is often applied after deblocking of reconstructed video data.
The conventional adaptive loop filter is only applied to these blocks where the filtering helps to improve performance. For other blocks that adaptive loop filtering does not help to improve performance, adaptive loop filtering is not applied. When the ALF is applied, a single filter SF is applied to the blocks in a slice. The single filter is selected from a group of filter candidates to achieve the best performance, such as minimum distortion, minimum rate or best R-D performance. Such ALF scheme is referred to as SF (single filter) ALF. Another ALF scheme (called QC_ALF) was proposed by Qualcomm (“Video coding technology proposal by Qualcomm Inc.”, Karczewicz et al., Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 1st Meeting: Dresden, D E, 15-23 Apr., 2010, Document: JCTVC-A121. According to QC_ALF, the ALF is applied to the deblocked video data on a block by block basis. For each block, the Sum-modified Laplacian Measure (SLM) SLM(i,j) of each pixel (i,j) of the block is computed. Each pixel of the block is then classified into multiple classes or categories according to the SLM measurement, and a respective ALF filter is selected for each pixel. While the QC_ALF scheme may improve performance over the SF ALF, the SLM computations for each pixel and filter switching from pixel to pixel will result in higher computational complexity and consume more power. It is desirable to develop an ALF scheme that provides the advantage of multiple filter choices for each region without high computational complexity and the need to switch filter from pixel to pixel.
During the ALF design in terms of region partition and mode decision, an optimization technique, called rate-distortion optimization (RDO), is often used to guide the region partition and mode decision. While the RDO technique achieves the best rate-distortion performance, it does not take into consideration of the system complexity, which may consume valuable system resources. It is desirable to use an optimization scheme that achieves the best performance in terms of rate, distortion, and complexity. Accordingly, the rate-distortion-complexity optimization (RDCO) technique is used for ALF design. The RDCO technique is also applied to sample adaptive offset (SAO) design to achieve the best rate-distortion-complexity performance.